Ubiquitous Memory Introspection ( Preliminary Manuscript )

Modern memory systems play a critical role in the performance of applications, but a detailed understanding of the application behavior in the memory system is not trivial to attain. It requires time consuming simulations of the memory hierarchy using long traces, and often using detailed modeling. It is increasingly possible to access hardware performance counters to measure events in the memory system, but the measurements remain coarse grained, better suited for performance summaries than providing instruction level feedback. The availability of a low cost, online, and accurate methodology for deriving fine-grained memory behavior profiles can prove extremely useful for runtime analysis and optimization of programs. This paper presents a new methodology for Ubiquitous Memory Introspection (UMI). It is an online and lightweight mini-simulation methodology that focuses on simulating short memory access traces recorded from frequently executed code regions. The simulations are fast and can provide profiling results at varying granularities, down to that of a single instruction or address. UMI naturally complements runtime optimizations techniques and enables new opportunities for memory specific optimizations. In this paper, we present a prototype implementation of a runtime system implementing UMI. The prototype is readily deployed on commodity processors, requires no user intervention, and can operate with stripped binaries and legacy software. The prototype operates with an average runtime overhead of 20% but this slowdown is only 6% slower than a state of the art binary instrumentation tool. We used 32 benchmarks, including the full suite of SPEC2000 benchmarks, for our evaluation. We show that the mini-simulation results accurately reflect the cache performance of two existing memory systems, an Intel Pentium 4 and an AMD Athlon MP (K7) processor. We also demonstrate that low level profiling information from the online simulation can serve to identify high-miss rate load instructions with a 77% rate of accuracy compared to full offline simulations that required days to complete. The online profiling results are used at runtime to implement a simple software prefetching strategy that achieves a speedup greater than 60% in the best case.